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by Michael T. Kane, Ph.D., Andrew A. Mroch, Ph.D., Douglas R. Ripkey, and Susan M. Case, Ph.D. 


hree recent studies sponsored by the 

New York State Board of Law Examiners 

have provided extensive information on 

the characteristics and performance of 
candidates taking the New York Bar Examination 
(NY bar exam). The first study, completed in 
2006,, examined the impact of an increase in the 
passing score for the NY bar exam from 660 to 665 
(implemented in July 2005) on candidate pass rates 
and projected what the pass rates would be if the 
passing score were increased to 670 or 675. These 
values of 660, 665, 670, and 675 translate to 132, 
133, 134, and 135, respectively, on the MBE scale. 
The analyses described in that report were based 
on the results for candidates who took the NY bar 
exam in July 2005 and who agreed to participate in 
the study. The sample of candidates who agreed to 
provide data included over 90% of the candidates 
who took the July 2005 NY bar exam and therefore 
provided good estimates of the characteristics of the 
population of candidates who took that examination 
and for various subgroups in that population. A 
second study’ reported a set of analyses similar 
to those from the first study but used data from 
a separate set of candidates who took the NY bar 
exam in February 2006. A third study* examined 
the subsequent performance in February 2006 and 
July 2006 of the July 2005 candidates who failed for 
the first time. The report for each of these studies is 


quite lengthy, and collectively, they contain a very 
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extensive set of results. 


This article summarizes some of the main results 
of the first and third studies and is necessarily quite 
selective. It focuses on three issues: (1) the initial 
performance of the domestic-educated candidates 
who took the New York bar exam for the first time 
in July 2005, (2) the persistence (in February and 
July of 2006) of the domestic-educated candidates 
who failed for the first time in July 2005, and (3) the 
pass rates for the domestic-educated first-time takers 
who failed in July 2005, as of February 2006 and as of 
July 2006. All three discussions include breakdowns 
by racial/ethnic group. For the complete results, 
please see the original studies available at the New 
York State Board of Law Examiners website (http:// 


www.nhybarexam.org). 


DATA 


The development of the data sets included in the 
three reports was made possible by the New York 
State Board of Law Examiners (the Board), the Law 
School Admission Council (LSAC), law schools, and 
candidates to the New York Bar. The Board (1) coordi- 
nated the collective efforts, (2) collected demo- 
graphic data (via an optional survey) from candi- 
dates when they applied to take the NY bar exam, 
and (3) assembled bar examination results from 
candidates after they completed the NY bar exam. 
LSAC provided Law School Admission Test (LSAT) 


scores, undergraduate grade-point averages, and 
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some demographic data for candidates who autho- 
rized release of these data. Law schools collaborated 
by providing law school GPAs from candidates who 
gave permission to share such information. All of 
these data were collected for the July 2005 data set 
for the first study. For the third study, the July 2005 
data set was expanded to include performance on the 
February 2006 and July 2006 administrations, along 
with cumulative pass rates, of candidates who failed 


the NY bar exam for the first time in July 2005. 


In analyzing the data, it was clear that the 
graduates of foreign law schools and the gradu- 
ates of domestic law schools constituted distinct 
populations with different demographic character- 
istics and different pass rates. Foreign-educated 
candidates constituted about 21% of the New 
York candidates who agreed to participate in the 
study, and as a group, they differed from domestic- 
educated candidates on a number of demographic 


variables. Foreign-educated candidates tended to 


- Not Indicated, 0.2% 


: Caucasian/White, 


have relatively low scores on the 
bar examination and relatively high 
failure rates, and as a result were 
much more likely than domestic- 
educated candidates to be repeating 
the bar examination. Also, under- 
eraduate GPAs, LSAT scores, and 
law school GPAs were not avail- 
able for foreign-educated candi- 
dates. Because of the substantial 
differences between these two pop- 
ulations and lack of availability of 
certain data for foreign-educated 
candidates, most of the analyses 
of candidate performance were re- 
ae ported separately for domestic- and 
foreign-educated candidates, and this 
article focuses on the performance of 


the domestic-educated candidates. 


Of the 7,252 July 2005 domestic-educated candi- 
dates, almost 50% were women, and slightly more 
than 50% were men. The great majority (over 70%) of 
the domestic-educated candidates were Caucasian/ 
White, 11.7% were Asian/Pacific Islander, 8.1% 
were Black/African American, 3.5% were Hispanic/ 
Latino, 1.2% were Puerto Rican, 0.4% were Chicano/ 
Mexican American, 0.2% were American Indian/ 
Alaskan Native, and 4.1% listed their race/ethnicity 
as “Other.” Figure 1, above, displays the percentages 
by race/ethnicity as a pie chart. 


Among the domestic-educated candidates, men 
were, on average, about half a year older than 
women when they graduated from law school (27.9 
vs. 27.4), and they were a little more than half a year 
older when they took the bar examination in July 
2005. Over 90% of the domestic-educated candidates 
taking the NY bar exam in July 2005 were taking it 


for the first time. 
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Although the domestic-educated first-time tak- 
ers were almost evenly split between women and 
men, the women/men ratios varied across racial/ 
ethnic groups. Men outnumbered women in the 
Caucasian/White group, but women outnumbered 
men in all of the other racial/ethnic groups, and in 
the Black/African American group, women outnum- 


bered men by almost two to one. 


PERFORMANCE OF CANDIDATES TAKING 
THE NEW YORK BAR EXAM IN JULY 2005 


Total scores on the NY bar exam are computed 
by combining three separate scaled and weighted 
scores from three separate components: the New 
York Essay Examination, which consists of five 
essay questions and one Multistate Performance Test 
(MPT) and has a weight of 50%, the Multistate Bar 
Examination (MBE), which includes 200 multiple- 
choice questions and has a weight of 40%, and the 
New York Multiple-Choice Test (NYMC), which 
includes 50 multiple-choice questions and has a 
weight of 10%. Using these weights, the scores on 
each component of the NY bar exam (the MBE, the 
Essay, and the NYMC) are combined and scaled to a 
1,000-point scale. The MBE scores are multiplied by 
5 and the essay scores and NYMC scores are scaled 
to the MBEx5 scores. 


The reliabilities of the components of the NY 
bar exam are all fairly high. MBE scores have a reli- 
ability of about .90. Multiple-choice tests typically 
have high reliabilities, and long multiple-choice tests 
(the MBE has 200 items) tend to have especially good 
reliabilities. The New York Multiple-Choice test is 
much shorter than the MBE, and as a result has a 
somewhat lower reliability, about .78. The essay 
component (including the MPT) has a reliability of 
about .80. The total score on the NY bar exam that 


results when the three components are combined 
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with the appropriate weights has a reliability of 
about .92. 


The variability in performance across groups 
(first-time takers and repeat takers, and the various 
racial/ethnic groups) was generally much larger 
than the differences across components of the exami- 
nation within any particular group. That is, groups 
that did relatively well on one component (e.g., the 
Essay) also tended to do well on the other two com- 
ponents (e.g., the MBE and the NYMC), and groups 
that didn’t do so well on one component didn’t do so 


well on the other components. 


The one noteworthy exception to this general- 
ization was a consistent tendency for women to do 
somewhat better than men on the essay component 
(about 2 points on the MBE scale) and for men to do 
somewhat better than women on the MBE (about 5 
points on the MBE scale); this effect was not very 
large on average, but it was observed consistently 
across racial/ethnic groups, and for first-time tak- 
ers and repeat takers. These two tendencies (women 
doing better on the essay component and men doing 
better on the MBE) tended to cancel out, and as a 
result, women and men did about equally well in 
terms of their total scores on the bar examination and 


their pass rates. 


The first-time takers did better on the examina- 
tion than the repeat takers. Candidates who had 
already taken the examination a number of times 
prior to July 2005 tended to have very low pass 
rates. The average score for domestic-educated first- 
time takers on the 0-1,000 scale was about 727 
(145.4 on the MBE scale), and the average total score 
for domestic-educated repeat takers was about 624 
(124.8 on the MBE scale), a difference of over one 
hundred points on the 1,000-point scale (about 21 
points on the MBE scale). 


Table 1 







JULY 2005 NEW YORK BAR EXAMINATION, SCORE fairly large sample sizes for the July 
MEANS FOR RACIAL/ETHNIC GROUPS, DOMESTIC- 2005 administration of the NY bar 
EDUCATED FIRST-TIME TAKERS exam. The Caucasian/White group 
had the highest average total score, 

MBE Essay | NYMC | Total 736 (147.2 on the MBE scale), fol- 


Scaled Scaled | Scaled | NY Bar lowed by the Asian/Pacific Islander 


Race/Ethnicity Score x 5* | Score Score Syome, the Hiseanieaanoeronp: 
Caucasian /White 736 737 730 736 and the Black/African American 
(n = 4,818) (72) (68) (75) (62) eroup, which had an average score 
Reina Pacha Ielandes 713 719 712 716 of 676 (135.2 on the MBE scale). 
(n = 740) (74) (73) (78) (65) The standard deviation (SD) for each 

673 679 671 676 eroup (shown in parentheses below 


pata ceiean the average score in Table 1) indi- 


aa (67) (67) (82) e7) cates the spread in the scores for 
Hispanics alino 700 707 702 703 the group, with about 68% of the 
(n = 214) (82) (68) (78) (67) eroup’s scores within one SD of the 
mean. Note that the difference be- 

Total** ted Me _ let tween the highest average score and 

(N = 6,585) (74) (71) (78) (64) the lowest average score across the 


_ eroups is 60 points. For the domestic- 
*MBE Scaled Score x 5 is the MBE scaled score (on a 0-200 scale) multiplied by 5 so 


that it is in a 0-1,000 range, similar to the total NY bar exam score. educated candidates repeating the 


**Total includes racial/ethnic groups with fewer than 20 candidates, which are not examination in July 2005, the dif- 


separately listed in the table. 
ferences across racial/ethnic groups 
NOTE: The numbers in parentheses are the standard deviations for each group on 


each bar exam component and total scores. were much less pronounced, with 

the averages ranging from about 631 

The average total score for domestic-educated (126.2 on the MBE scale) to about 
repeat takers decreased systematically as the number 613 (122.6 on the MBE scale). 


of previous attempts increased. Domestic-educated 


second-time takers had an average of about 635 (127 A second point to notice in Table 1 is the relative 


on she MBE ceale), (nd tie takew a tencvene consistency in each group’s performance across the 
of about 627 (125.4 on the MBE scale), and fourth- 
time takers had an average total score of about 620 


(124 on the MBE scale). 


three subtests. The largest difference across the MBE, 
the Essay, and the NYMC for any of the groups is 8 
points (1.6 points on the MBE scale), which is much 


smaller than the between-group differences on the 


the NY bar exam for the first time in July 2005, the 


: — Figure 2 presents essentially the same results as 
racial/ethnic groups exhibited large differences in 5 P y 


their average bar exam scores. Table 1 reports on the EDIE DENG UC OE etna able Poe er aver 


age scores for each group is relatively flat, and the 
average bar exam scores for the four groups with 5 aa 


variation across the subtests is much smaller than the 
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Figure 2 
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range of differences across the groups. The groups 
with relatively high or low total scores had corre- 


spondingly high or low scores on all of the subtests. 


As noted earlier, the difference in average total 
bar scores between men and women was relatively 
small. For domestic-educated first-time takers, the 
average total bar examination score was about 731 
(146.2 on the MBE scale) for men and about 724 
(144.9 on the MBE scale) for women. 


The pass rates for the July 2005 administration 
reflected the average scores for different groups. 
The differences in pass rates were quite large across 
the racial/ethnic groups, with the Caucasian /White 
first-time takers having the highest pass rate (about 
87%), and the Black/ African American first-time tak- 
ers having the lowest pass rate (about 54%), while 
the differences between men and women were quite 


small (about 85% for men and 82% for women). 


Among the domestic-educated candidates, the 
repeat takers, as a whole, had much lower pass rates 
(about 23%) than the first-time takers (83%), and the 


repeat takers’ pass rates tended to get lower as the 
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Total NY Bar 


number of previous attempts increased. Those who 
were repeating for the first time had higher pass 
rates (about 32%) than those repeating for the second 
time (about 26%), who in turn had higher pass rates 
than those who were repeating for the third or more 


times. 


PERSISTENCE RATES OF CANDIDATES 
WHO FAILED IN JULY 2005 


As expected, the domestic-educated candidates who 
failed the NY bar exam in July 2005 tended to retake 
the bar exam in 2006. Of the 1,241 domestic-educated 
candidates who failed the NY bar exam for the first 
time in July 2005, a total of 1,056 were identified as 
having retaken a bar examination in New York or 
another jurisdiction by July 2006. So at least 85% 
of the first-time-failing candidates repeated a bar 


examination by the following July. 


Persistence is reported in Figure 3 for the total 
group and for the four racial/ethnic groups that 
had substantial numbers of candidates who failed 
for the first time in July 2005. Each bar corresponds 


to a particular group of candidates, with pieces of 


NEw YORK BAR EXAMINATION, PERCENTAGES REPEATING July 2006. This percentage is 
IN FEBRUARY 2006 OR IN JULY 2006 FOR RACIAL/ETHNIC substantially smaller than the 
GROUPS, DOMESTIC-EDUCATED CANDIDATES FAILING FOR percentage of those repeating 
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the bar corresponding to percentages not repeating, 
repeating in February 2006 in New York, repeating 
in February 2006 outside New York, repeating in 
July 2006 in New York, and repeating in July 2006 


outside New York. 


Overall, at least three-quarters of the domestic- 
educated candidates who failed the NY bar exam for 
the first time in July 2005 repeated a bar examination 
in February 2006. Although there are some differ- 
ences in the percentages reported across groups, 
these differences are relatively small.’ The Hispanic / 
Latino group had the highest persistence level in 
February 2006, but the difference between this 
eroup’s level and the next-highest level is not large 


and may be due to sampling variability. 


About 10% of the domestic-educated candi- 
dates who failed the NY bar exam for the first 


time in July 2005 repeated a bar examination in 


as of July 2006 after failing 
for the first time in July 2005, 
but were identified as taking 
a bar examination in another jurisdiction, 46 (or just 
under half) took the New Jersey Bar Examination. 
Some candidates who took both the NY bar exam 
and the New Jersey bar exam in July 2005 could have 
failed in New York and passed in New Jersey; such 
candidates would be classified in the study as hav- 
ing failed, and assuming that they did not take a bar 
examination in 2006, they would also be classified in 


the study as not persisting. 


SCORE CHANGES BETWEEN JULY 2005 
AND FEBRUARY 2006 


Domestic-educated first-time takers who failed in 
July 2005 and retook the examination in February 
2006 improved their scores by about 44 points (8.8 
on the MBE scale) on average. However, some candi- 
dates achieved much larger increases and some can- 


didates suffered declines in their scores between July 
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Figure 4 
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2005 and February 2006. The average scores increased 
on all three of the subtests (MBE, Essay, and NYMC), 
with the largest increase in the Essay scores and the 


smallest increase in the NYMC scores. 


Figure 4 shows the relationship between individ- 
ual candidate scores in July 2005 (on the horizontal 
or X axis) and scores on the February 2006 admin- 
istration (on the vertical or Y axis) for candidates 
who took the NY bar exam for the first time in July 
2005. Figure 4 illustrates a number of issues. First, 
there are no points corresponding to July 2005 scores 
ereater than 665, or 133 on the MBE scale (i.e., to the 
right of the dotted vertical line); candidates with July 
2005 scores greater than 665 would have passed and 
therefore would not have repeated the NY bar exam 
in February 2006. 


Second, although there is a clear tendency for 
candidates with relatively high scores in July 2005 
to get relatively high scores in February 2006, and 


for candidates with relatively low scores in July 2005 
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Note: The diagonal line indi- 
cates the location of points for 
candidates with the same scores 
in July 2005 and February 2006. 
Points above this diagonal line 
indicate candidates whose scores 
improved between July 2005 and 
February 2006, and points below 
this diagonal indicate candidates 
whose scores decreased. 














to get low scores in February 2006, the relationship 
is far from perfect. Some candidates with relatively 
high scores in July 2005 received relatively low 
scores in February 2006, and some candidates with 
relatively low scores in July 2005 received fairly high 
scores in February 2006. The diagonal line in Figure 
4 indicates where the points would fall if the candi- 
dates received exactly the same scores in July 2005 
and February 2006. The fact that most of the points 
are above the diagonal line indicates that most of the 
candidates who failed the NY bar exam for the first 
time in July 2005 received higher scores when they 
repeated the examination in February 2006, and in 


many cases the increases were quite substantial. 


The dotted horizontal line in Figure 4 represents 
the passing score of 665 (133 on the MBE scale) for 
February 2006. For each of the points above this line, 
the candidate would have failed in July 2005 and 
passed in February 2006. The chances of passing in 


February 2006 are much better for candidates who 


Figure 5 
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had scores close to the passing score in July 2005 than 
they are for candidates who had scores far below the 


passing score in July 2005. 


For the domestic-educated first-time takers who 
failed in July 2005, the average scores in July 2005 
and in February 2006 were fairly consistent across 
the racial/ethnic categories, as illustrated in Figure 5; 
scores increase for each group and do so at a similar 
rate. The differences in change scores among these 
groups were not large enough to be statistically 
meaningful and were quite small compared to the 
differences in average scores across racial/ethnic 
groups that were found for all domestic-educated 


first-time takers in July 2005. 


PASS RATE CHANGES BETWEEN JULY 
2005 AND JULY 2006 
The candidates who failed the NY bar exam for the 


first time in July 2005 had opportunities to retake 
the NY bar exam in February and July 2006, and the 


pass rates of about 
57% in February 2006 
and about 32% in 
July 2006. As a result 
of the high persistence rates of the first-time-failing 
candidates and their substantial pass rates when 
retaking the bar exam, the cumulative pass rates for 
the July 2005 first-time takers increased from about 
83% in July 2005, to almost 90% in February 2006, 
and to just over 91% in July 2006. The increase was 
quite substantial from July 2005 to February 2006, 
but the additional increase from February 2006 to 


July 2006 was relatively modest. 


The initial pass rate was slightly higher for 
domestic-educated first-time-taking men than for 
similarly situated women candidates (84.0% vs. 
82.2%), and this difference of 1.8 percentage points 
shrank a bit between July 2005 and July 2006. By 
February 2006, the difference in cumulative pass 
rates was 1.4 percentage points (90.4% vs. 89.0%), 
and by July 2006, the difference was 1.0 percentage 
point (91.7% vs. 90.7%). 
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Figure 6 
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Among the first-time takers who failed in July 
2005, the different racial/ethnic groups were similar 
in their persistence rates, in their average scores in 
July 2005, and in their average improvement in scores 
between July 2005 and February 2006. As a result, 
the differences in pass rates observed across racial/ 
ethnic groups for the first-time takers in July 2005 
diminished as the failing candidates had a chance to 
repeat the exam in February 2006 and then in July 
2006. However, the differences in cumulative pass- 
ing rates across racial/ethnic groups were still fairly 
large as of July 2006. While the cumulative pass rates 
for the Caucasian/White first-time takers increased 
by 6.6 percentage points, from 86.8% in July 2005 to 
93.4% as of July 2006, and the cumulative pass rates 
for the Black/African American first-time takers 
increased by 20.9 percentage points, from 54.2% in 
July 2005 to 75.1% as of July 2006, the cumulative 
pass rate for the Black/African American group was 


still over 18 percentage points lower than that of the 
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chance of contributing to the 
numerators because they did 
not take the New York bar 
exam in 2006. Some of these 
candidates were identified as 
taking bar examinations in 
different jurisdictions. Some may have already been 
admitted to the bar in other jurisdictions in July 2005 
without being identified as having done so. Some 
may have decided not to practice law in the United 
States. If the non-persisters were removed from the 
calculations, the denominators would get smaller, 
and the pass rates would increase. The total pass 
rate for first-time takers who failed in July 2005 and 
repeated in February 2006 was 89.5%, and would 
increase to 93.1% if the non-persisters were excluded 
from the analysis. The total pass rate as of July 2006 
was 91.1% and would increase to 94.7%. The pass 
rates increase by about 3.6 percentage points if the 
non-persisters are excluded from the analysis. These 
adjusted pass rates are probably overestimates of the 
pass rates and the original unadjusted pass rates are 
almost certainly underestimates. So the total pass rate 
as of July 2006 is probably between 91.1% and 94.7%. 


The various subgroups all exhibited increases 
in pass rates when the non-persisters were removed 
from the calculations of the cumulative pass rates 
as of February 2006 and July 2006. The Caucasian/ 
White pass rate as of July 2006 increased from 93.4% 
to 96.3%. The Asian/Pacific Islander pass rate as of 
July 2006 increased from 89.8% to 93.9%. The Black/ 
African American pass rate as of July 2006 increased 
from 75.1% to 82.6%. The Hispanic/Latino pass rate 
as of July 2006 increased from 84.8% to 89.3%. 


CHARACTERISTICS OF CANDIDATES 
WHO FAILED IN JULY 2005 AND 
PASSED IN 2006 


In examining the differences between July 2005 first- 
time-failing candidates who persisted and passed in 
2006, those who persisted and failed, and those who 
did not persist, we found that the candidates who 
had July 2005 bar exam scores that were close to 665 
had a better chance of passing the NY bar exam if 
they retook the bar exam in February 2006 or July 
2006 than those with relatively low bar exam scores 
in July 2005. Also, July 2005 first-time-failing candi- 
dates who persisted and passed within a year of the 
first exam tended to have done somewhat better on 
a range of indicators of academic preparedness than 
those who persisted and failed. The candidates who 
persisted and passed in 2006 had higher average 
undergraduate GPAs, higher average LSAT scores, 
higher average law school grades (using two scaling 
methods), and higher average bar exam scores on 
their first attempt in July 2005 than the candidates 
who persisted but did not pass in 2006. 


Those who did not persist (i.e., were not identi- 
fied as having taken a bar exam in 2006) exhibited 
average academic achievement profiles that were 
not as good as those of candidates who persisted 
and passed, but were better than those of candidates 


who persisted and failed. It would be interesting and 


useful to find out more about why candidates did 


not persist in 2006. 


RELATIONSHIP BETWEEN CUMULATIVE 
PASS RATES AND PRIOR ACHIEVEMENT 


Analyses of the relationship between a candidate’s 
chances of passing the bar examination in July 2005, 
by February 2006, and by July 2006 and his or her 
prior achievement’ suggest that performance in law 
school as measured by law school GPA is strongly 
related to the probability of passing the NY bar 
exam on the first try and to the probability of pass- 
ing within a year of the first try. In general, perfor- 
mance in law school, as measured by law school 
GPA, was the best predictor of performance on the 
bar examination, accounting for about 40% to 47% 
of the variance (or variability) in bar examination 
scores (depending on the specific model employed).’ 
Adding information about undergraduate GPA and 
LSAT scores (in addition to law school GPAs) to the 
prediction equations improved the accuracy of the 
prediction to cover about 56% of the variance in bar 


examination scores. 


Since the bar examination is intended to assess 
each candidate’s readiness for practice in terms of 
his or her competence in applying basic legal prin- 
ciples to practice situations, and since law schools 
presumably assess these skills in grading students, 
the existence of a positive relationship between law 
school GPA and performance on the bar examination 
was not surprising. Because law school GPA is, in 
turn, related to performance on measures of readi- 
ness for law school (LSAT scores and undergraduate 
GPAs), the positive relationship between bar exam 
scores and scores on measures of readiness for law 
school was also not very surprising. It seems that the 
candidates who do relatively well on the LSAT and 


undergraduate GPA tend to do relatively well in law 
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school, and subsequently tend to do relatively well 


on the bar examination. 


CONCLUSIONS 


The domestic-educated first-time takers who failed 
in July 2005 generally retook the NY bar exam in 
February 2006 and/or July 2006. As a result of their 
high persistence rates and fairly high pass rates when 
retaking the bar exam, the cumulative pass rates for 
the July 2005 first-time takers increased from about 
83% in July 2005 to almost 90% in February 2006 and 
to just over 91% in July 2006. 


Although we found large differences in scores 
and pass rates across different racial/ethnic groups 
for the domestic-educated first-time takers in July 
2005, the pass rates for the July 2005 first-time-failing 
candidates when they repeated in February 2006 
were quite similar across racial/ethnic group and 


gender categories. 


The differences in pass rates across various sub- 
groups diminished as the failing candidates had a 
chance to repeat the NY bar exam in February and 
July 2006, but the differences in cumulative pass 
rates across some subgroups were still fairly large as 
of July 2006. By July 2006, the cumulative pass rate 
for the Caucasian/White group was between 93.4% 
and 96.3% and that for the Black/African American 
eroup was between 75.1% and 82.6%. 


In this article, we summarized some of the 
results of three studies sponsored by the New 
York State Board of Law Examiners. Each study 
contains a wealth of information regarding the 
characteristics and performance of candidates to 
the New York bar, and the three studies provide an 
example of successful collaboration and data sharing 
among several groups of stakeholders, including 
candidates to the New York bar. The endnotes below 
provide links to the studies, which are posted on the 
Board’s website. B 
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(GLOSSARY 


Mean: The average of a set of scores. Technically, the 
mean is defined as the sum of the scores divided by the 


number of scores. 


Pass rate: The percentage of a group of candidates that 
would pass at a particular passing score. 


Passing score: The total numerical score on an exami- 
nation that a candidate has to achieve in order to pass 
the examination. 


Reliability: The consistency or repeatability of the 
scores produced by a measurement procedure; the pre- 
cision in the scores yielded by a measurement instru- 
ment. Reliability is defined as the variance in “true” 
scores divided by the variance in observed scores. The 
observed score for an individual is assumed to consist 
of the true score plus an error component, and therefore 
the variance in observed scores is equal to the variance 
in the true scores plus the error variance. So the reliabil- 
ity is always between 0.0 and 1.0. Reliability can also 
be interpreted as a correlation coefficient, with values 
between 0.0 and 1.0. Higher values for reliability reflect 
ereater precision and less random error, and low values 
for reliability reflect a higher proportion of random 


error and therefore less precision. 


Scaling: The process of transforming a set of scores 
on a test so that they have the same mean (or average) 
and same standard deviation (or spread) as scores on 
another test. The intent of scaling is to make the scores 
comparable in the sense that an average score on both 
tests would be about the same, the highest scores on 
both tests are about the same, and the lowest scores 
on both tests are about the same. One commonly used 
scaling process converts temperature from Fahrenheit 


to Celsius. 


Standard deviation (SD): A measure of the spread 
in a set of scores. Technically, the standard deviation 
is defined as the square root of the average squared 
deviation from the mean. About 68% of the scores in 
a distribution will be within one standard deviation 


of the mean. 


Variance: A measure of the spread in a set of scores, 


equal to the square of the standard deviation. 
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These results on the persistence rates for the racial/ethnic 
groups are consistent with the results of the LSAC National 
Longitudinal Bar Passage Study (Wightman, 1998). Although 
some data in the LSAC report were interpreted as indicating 
that Black/African American candidates had substantially 
lower persistence rates than the Caucasian/White and Asian / 
Pacific Islander candidates (suggesting the existence of a “per- 
sistence gap”), this conclusion resulted from a misinterpreta- 
tion of the results. Basically, ratios that were not persistence 
rates were interpreted as if they were persistence rates. See 
the “President’s Page” by Erica Moeser and the attached col- 
umn reprinted from LSAC’s LAW SERVICES REPoRT, No. 2000-4, 
December 2000 by Rennard J. Strickland, in the February 2001 
BAR EXAMINER for a more detailed explanation. 


The data for this analysis included a subsample of domestic- 
educated first-time takers who failed the NY bar exam in July 
2005 and had undergraduate GPAs, LSAT scores, law school 
GPAs, and bar examination scores available. 


A predictor that accounts for 40 to 50 percent of the variance 
in an outcome measure would generally be considered a 
strong predictor. For comparison, LSAC scores account for 
about 20 to 25 percent of the variance in first-time bar exami- 
nation scores, and undergraduate GPA accounts for about 
10 to 15 percent of the variance in first-time bar examination 
scores. The SAT scores, used in college admissions, account 
for about 25 percent of the variance in first-year GPAs in a 
typical college. 
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